Characterizing Video-based Activity using a 3D Structure Tensor

This paper describes a structure tensor based method to characterize activity in videos. Based on the relationship observed between structure tensors and covariance matrices, the distance between structure tensor matrices is computed by either a generalized eigenvalue or a Riemannian manifold. A histogram of distances between structure tensor matrices is constructed to serve as a signature for motion clutter. This method differs from the usual measurement using smallest eigenvalue. It is shown, experimentally, that the distance between structure tensors evaluates uncertainty along different orientations. The distribution for specific activities exhibit a stable histogram over time. That is, experimental results show that there is coherent correlation between activity pattern and corresponding probability density function of distance between structure tensor matrices.